Reducing watermark perceptibility and extending detection distortion tolerances

ABSTRACT

The present disclosures relates generally to digital watermarking and data hiding. One claim recites an apparatus comprising: memory for storing data representing video; one or more electronic processors programmed for: embedding a first watermark signal in a first portion of the data, the first watermark signal comprising a first signal polarity and corresponding to first detection preconditioning; embedding a second watermark signal in a second portion of the data, the second watermark signal comprising a second signal polarity that is inversely related to the first signal polarity and corresponding to seconding detection preconditioning; controlling provision of the watermarked video for display in real time, in which temporal averaging of the first watermark signal and second watermark signal over time conceals the first watermark signal and the second watermark signal from a human observer of the video. Of course, other claims are provided too.

RELATED APPLICATION DATA

This application is a continuation of U.S. application Ser. No.13/042,212, filed Mar. 7, 2011 (now U.S. Pat. No. 8,477,990), whichclaims the benefit of U.S. Provisional Application No. 61/311,218, filedMar. 5, 2010. This application is related to U.S. patent applicationSer. No. 12/634,505, filed Dec. 9, 2009 (published as US 2010-0150396A1); Ser. No. 12/337,029, filed Dec. 17, 2008 (published as US2010-0150434 A1); and Ser. No. 12/640,386, filed Dec. 17, 2009 (now U.S.Pat. No. 8,175,617). Each of the above patent documents is herebyincorporated herein by reference in its entirety.

TECHNICAL FIELD

The disclosure relates generally to steganographic data hiding anddigital watermarking.

BACKGROUND AND SUMMARY

The term “steganography” generally means data hiding. One form of datahiding is digital watermarking. Digital watermarking is a process formodifying media content to embed a machine-readable (ormachine-detectable) signal or code into the media content. For thepurposes of this application, the data may be modified such that theembedded code or signal is imperceptible or nearly imperceptible to auser, yet may be detected through an automated detection process. Mostcommonly, digital watermarking is applied to media content such asimages, audio signals, and video signals.

Digital watermarking systems may include two primary components: anembedding component that embeds a watermark in media content, and areading component that detects and reads an embedded watermark. Theembedding component (or “embedder” or “encoder”) may embed a watermarkby altering data samples representing the media content in the spatial,temporal or some other domain (e.g., Fourier, Discrete Cosine or Wavelettransform domains). The reading component (or “reader” or “decoder”) mayanalyze target content to detect whether a watermark is present. Inapplications where the watermark encodes information (e.g., a message orpayload), the reader may extract this information from a detectedwatermark.

A watermark embedding process may convert a message, signal or payloadinto a watermark signal. The embedding process may then combine thewatermark signal with media content and possibly another signals (e.g.,an orientation pattern or synchronization signal) to create watermarkedmedia content. The process of combining the watermark signal with themedia content may be a linear or non-linear function. The watermarksignal may be applied by modulating or altering signal samples in aspatial, temporal or some other transform domain.

A watermark encoder may analyze and selectively adjust media content togive it attributes that correspond to the desired message symbol orsymbols to be encoded. There are many signal attributes that may encodea message symbol, such as a positive or negative polarity of signalsamples or a set of samples, a given parity (odd or even), a givendifference value or polarity of the difference between signal samples(e.g., a difference between selected spatial intensity values ortransform coefficients), a given distance value between watermarks, agiven phase or phase offset between different watermark components, amodulation of the phase of the host signal, a modulation of frequencycoefficients of the host signal, a given frequency pattern, a givenquantizer (e.g., in Quantization Index Modulation) etc.

The present assignee's work in steganography, data hiding and digitalwatermarking is reflected, e.g., in U.S. Pat. Nos. 6,947,571; 6,912,295;6,891,959, 6,763,123; 6,718,046; 6,614,914; 6,590,996; 6,408,082;6,122,403 and 5,862,260, and in published specifications WO 9953428 andWO 0007356 (corresponding to U.S. Pat. Nos. 6,449,377 and 6,345,104).Each of these patent documents is hereby incorporated by referenceherein in its entirety. Of course, a great many other approaches arefamiliar to those skilled in the art. The artisan is presumed to befamiliar with a full range of literature concerning steganography, datahiding and digital watermarking.

One combination recites a method comprising: obtaining data representingvideo; using one or more electronic processors, embedding a firstwatermark signal in a first portion of the data, the first watermarksignal comprising a first signal polarity; using one or more electronicprocessors, embedding a second watermark signal in a second portion ofthe data, the second watermark signal comprising a second signalpolarity that is inversely related to the first signal polarity;rendering the watermarked video in real time, in which due to temporalaveraging of the first watermark signal and second watermark signal overtime, the first watermark signal and the second watermark signal arehidden from a human observer of the video.

Another combination includes a method comprising: obtaining datarepresenting video; using one or more electronic processors, embedding awatermark signal in a first portion of the data, the embedding using afirst embedding bump size; using one or more electronic processors,embedding a watermark signal in a second portion of the data, theembedding using a second embedding bump size, in which the firstembedding bump size corresponds with a first optimal detection rangedistance when capturing optical scan data associated with the video asit is being rendered on a display, and the second embedding bump sizecorresponds with a second, larger optimal detection range distance whencapturing optical scan data associated with the video as it is beingrendered on the display.

Yet another combination includes a method comprising: obtaining awatermark signal; using one or more programmed electronic processors,embedding a watermark signal in a first portion of a video signal;preconditioning the watermark signal in a first manner to allow expandeddetection of said preconditioned watermark signal in the face of firstdistortion; using one or more programmed electronic processors,embedding the watermark signal preconditioned in the first manner in asecond portion of the video signal; preconditioning the watermark signalin a second manner to allow expanded detection of said preconditionedwatermark signal in the face of second distortion; using one or moreprogrammed electronic processors, embedding the watermark signalpreconditioned in the second manner in a third portion of the videosignal.

Still another claim recites a method comprising: receiving datarepresenting video captured from a video display; searching the data forhidden indicia, the indicia providing information to allow adetermination of video capture distance and video capture perspective;upon encountering the hidden indicia, using the information to warp thedata to compensate for distortion caused by video capture distance orvideo capture perspective; provided the warped data to an electronicprocessor programmed as a steganographic indicia decoder, said decoderanalyzes the warped data to decode steganographic indicia hidden invideo captured from the video display.

Further combinations, aspects, features and advantages will become evenmore apparent with reference to the following detailed description andaccompanying drawing.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 represents a color image.

FIG. 2 represents a first color channel (‘a’ channel) of the color imagerepresentation shown in FIG. 1.

FIG. 3 represents a second color channel (‘b’ channel) of the colorimage representation shown in FIG. 1.

FIG. 4 is a representation of the sum of the first color channel of FIG.2 and the second color channel of FIG. 3 (e.g., a+b).

FIG. 5 is a graph showing a histogram standard deviation of FIG. 4.

FIG. 6 is a representation of the difference between the first colorchannel of FIG. 2 and the second color channel of FIG. 3 (a−b).

FIG. 7 is a graph showing a histogram standard deviation of FIG. 6.

FIG. 8 is an image representation of the difference between the firstcolor channel of FIG. 2 (including a watermark signal embedded therein)and the second color channel of FIG. 3 (including the watermark signalembedded therein).

FIG. 9 is a graph showing a histogram standard deviation of FIG. 8.

FIGS. 10 a and 10 b are block diagrams showing, respectively, anembedding process and a detection process.

FIG. 11 is a diagram showing watermarks embedded in first and secondvideo frames.

FIG. 12 is a diagram showing inversely related watermark signals in twovideo frames.

FIG. 13 is a diagram showing image capture of rendered video.

FIG. 14 shows a representation of a watermark signal.

FIG. 15 shows a preconditioned version of the FIG. 14 watermark signal.

FIG. 16 shows a graphical target, which can be used to provideinformation associated with image capture distance and image captureperceptive angle.

DETAILED DESCRIPTION

Some aspects of the following disclosure discuss a digital watermarkingtechnique that utilizes at least two chrominance channels (also called“color planes,” “color channels” and/or “color direction”). Chrominanceis generally understood to include information, data or signalsrepresenting color components of an image or video. In contrast to acolor image or video, a grayscale (monochrome) image or video has achrominance value of zero.

Media content that includes a color image (or color video) isrepresented in FIG. 1. An industry standard luminance and chrominancecolor space is called “Lab” (for Lightness (or luminance), plus ‘a’ and‘b’ color channels) that can be used to separate components of imagesand video. FIG. 2 is an ‘a’ channel representation of FIG. 1 (shown ingrayscale), and FIG. 3 is a ‘b’ channel representation of FIG. 1 (shownin grayscale). Of course, our inventive methods and apparatus will applyto and work with other color schemes and techniques as well. Forexample, alternative luminance and chrominance color schemes include“Yuv” (Y=luma, and ‘u’ and ‘v’ represent chrominance channels) and“Ycc.” (also a dual chrominance space representation).

Let's first discuss the additive and subtractive effects on FIGS. 2 and3. FIG. 4 illustrates a representation of the result of adding the ‘a’channel (FIG. 2) with the ‘b’ channel (FIG. 3). FIG. 6 illustrates arepresentation of the result of subtracting the ‘b’ channel (FIG. 3)from the ‘a’ channel (FIG. 2). The result of subtracting the ‘b’ channelfrom the ‘a’ channel yields reduced image content relative to adding thetwo channels since the ‘a’ and ‘b’ color planes have correlated imagedata in the Lab scheme. (In typical natural imagery, the ‘a’ and ‘b’chrominance channels tend to be correlated. That is to say where ‘a’increases, ‘b’ also tends to increase. One measure of this is to measurethe histogram of the two chrominance planes when they are added (seeFIG. 5), and compare that to the histogram when the two color planes aresubtracted (see FIG. 7). The fact that the standard deviation of FIG. 7is about half that of FIG. 5 also supports this conclusion, andillustrates the reduction in image content when ‘b’ is subtracted from‘a’) In this regard, FIG. 4 provides enhanced or emphasized imagecontent due to the correlation. Said another way, the subtraction of theFIG. 3 image from FIG. 2 image provides less image interference orreduces image content. The histogram representations of FIG. 4 and FIG.6 (shown in FIGS. 5 and 7, respectively) further support thisconclusion.

Now let's consider watermarking in the context of FIGS. 2 and 3.

In a case where a media signal includes (or may be broken into) at leasttwo chrominance channels, a watermark embedder may insert digitalwatermarking in both the ‘a’ color direction (FIG. 2) and ‘b’ colordirection (FIG. 3). This embedding can be preformed in parallel (ifusing two or more encoders) or serial (if using one encoder). Thewatermark embedder may vary the gain (or signal strength) of thewatermark signal in the ‘a’ and ‘b’ channel to achieve improved hidingof the watermark signal. For example, the ‘a’ channel may have awatermark signal embedded with signal strength that greater or less thanthe watermark signal in the ‘b’ channel. Alternatively, the watermarksignal may be embedded with the same strength in both the ‘a’ and ‘b’channels. Regardless of the watermark embedding strength, watermarksignal polarity is preferably inverted in the ‘b’ color plane relativeto the ‘a’ color plane. The inverted signal polarity is represented by aminus (“−”) sign in equations 1 and 2.WMa=a(channel)+wm  (1)WMb=b(channel)−wm  (2)

WMa is a watermarked ‘a’ channel, WMb is a watermarked ‘b’ channel, andwm represents a watermark signal. A watermarked color image (including Land WMb and WMa) can be provided, e.g., for printing, digital transferor viewing.

An embedded color image is obtained (from optical scan data, memory,transmission channel, etc.), and data representing the color image iscommunicated to a watermark detector for analysis. The detector (or aprocess, processor or electronic processing circuitry used inconjunction with the detector) subtracts WMb from WMa resulting in WMresas shown below:WMres=WMa−WMb  (3)WMres=(a+wm)−(b−wm)  (4)WMres=(a−b)+2*wm  (5)

This subtraction operation yields reduced image content (e.g., FIG. 6)as discussed above. The subtraction or inverting operation of the colorchannels also emphasizes or increases the watermark signal (2*wm),producing a stronger watermark signal for watermark detection. Indeed,subtracting the color channels increases the watermark signal-to-mediacontent ratio: WMres=(a−b)+2*wm.

FIG. 8 illustrates the result of equation 5 (with respect to watermarkedversions of FIG. 2 and FIG. 3). As shown, the perceptual “graininess” or“noise” in the image corresponds to the emphasized watermark signal. Theimage content is also reduced in FIG. 8. A histogram representation ofFIG. 8 is shown in FIG. 9 and illustrates a favorable reduction of imagecontent.

A watermark detector may extract or utilize characteristics associatedwith a synchronization signal (if present) from a frequency domainrepresentation of WMres. The detector may then use this synchronizationsignal to resolve scale, orientation, and origin of the watermarksignal. The detector may then detect the watermark signal and obtain anymessage or payload carried thereby.

To even further illustrate the effects of improving the watermarksignal-to-media content ratio with our inventive processes and systems,we provide some additive and subtractive examples in the content ofwatermarking.

For the following example, a watermark signal with the same polarity isembedded in each of the ‘a’ color channel and the ‘b’ color channel. Thesame signal polarity is represented by a plus (“+”) sign in equations 6and 7.WMa=a+wm  (6)WMb=b+wm  (7)

WMa is a watermarked ‘a’ channel, WMb is a watermarked ‘b’ channel, andwm represents a watermark signal. A watermarked color image (including Land WMb and WMa) can be provided, e.g., for printing, digital transferor viewing.

An embedded color image is obtained, and data representing the colorimage is communicated to a watermarked detector for analysis. Thedetector (or a process, processor, or electronic processing circuitryused in conjunction with the detector) adds the ‘a’ and ‘b’ colorchannels to one another (resulting in WMres) as shown below:WMres=WMa+WMb  (8)WMres=(a+wm)+(b+wm)  (9)WMres=(a+b)+2*wm  (10)

This addition operation results in increased image content (e.g., FIG.4). Indeed, image interference during watermark detection will begreater since the two correlated ‘a’ and ‘b’ color channels tend toreinforce each other.

By way of further example, if WMb is subtracted from WMa (with watermarksignals having the same polarity), the following results:WMres=WMa−WMb  (11)WMres=(a+wm)−(b+wm)  (12)WMres=(a−b)+≈0*wm  (13)

A subtraction or inverting operation in a case where a watermark signalincludes the same polarity decreases image content (e.g., FIG. 4), butalso significantly decreases the watermark signal. This may result inpoor—if any—watermark detection.

FIGS. 10 a and 10 b are flow diagrams illustrating some relatedprocesses and methods. These processes may be carried out, e.g., via acomputer processor, electronic processing circuitry, printer, handhelddevice such as a smart cell phone, etc.

With reference to FIG. 10 a, a color image (or video) is obtained andseparated into at least two (2) color channels or planes (10). Awatermark signal is determined for the color image or video (12). Ofcourse, the watermark signal for the color image or video may bedetermined prior to or after color plane separation. The determinedwatermark signal is embedded in a first of the color planes (14). Aninverse polarity version of the watermark signal is embedded in a secondcolor plane. The color planes are recombined (perhaps with datarepresenting luminance) to form a composite color image.

With reference to FIG. 10 b, a watermarked color image or video isobtained or received (11). The color image (or video) has or can beseparated into at least two (2) color planes or channels (13). A firstcolor plane includes a watermark signal embedded therein. A second colorplane includes the watermark signal embedded therein with a polaritythat is inversely related to the watermark signal in the first colorplane. The watermarked second color plane is subtracted from thewatermarked first color (15). The result of the subtraction is analyzedto detect the watermark signal. A detected watermark message, signal orpayload can be provided (19), e.g., to a remote database to obtainrelated metadata or information, to a local processor, for display, to arights management system, to facilitate an online transaction, etc.

In addition to the Lab color scheme discussed above, a watermark signalmay be embedded in color image (or video) data represented by RGB, Yuv,Ycc, CMYK or other color schemes, with, e.g., a watermark signalinserted in a first chrominance direction (e.g., red/green direction,similar to that discussed above for the ‘a’ channel) and a secondchrominance direction (e.g., a blue/yellow direction, similar to thatdiscussed above for the ‘b’ channel). For watermark signal detectionwith an alternative color space, e.g., an RGB or CMYK color space, animage can be converted to Lab (or other color space), or appropriateweights of, e.g., RGB or CMY channels, can be used. For example, thefollowing RGB weights may be used to calculate ‘a’−‘b’: ChrominanceDifference=0.35*R−1.05*G+0.70*B+128, where R, G and B are 8-bitintegers.

Further Considerations of Video

The human contrast sensitivity function curve shape with temporalfrequency (e.g., relative to time) has a very similar shape to thecontrast sensitivity with spatial frequency.

Successive frames in a video are typically cycled at about at least 60Hz to avoid objectionable visual flicker. So-called “flicker” is due tothe high sensitivity of the human visual system (HVS) to high temporalfrequency changes in luminance. The human eye is about ten (10) timesless sensitive to high temporal frequency chrominance changes.

Consider a video sequence with frames as shown in FIG. 11. A chrominancewatermark can be added to frame 1 per the above description for images.In a similar way, a watermark is added to frame 2 but the polarity isinverted as shown in FIG. 11.

In order to recover the watermark, pairs of frames are processed by awatermark detector, and the ‘a’ channels are subtracted from each otheras shown below.Det_(—) a=(a1+wm)−(a2−wm)=(a1−a2)+2*wm  (14)

Det_a refers to watermark detection processing of the ‘a’ channel.Because of the temporal correlation between frames, the image content inequation 14 is reduced while the watermark signal is reinforced.

In a similar way the ‘b’ channels are also subtracted from each otherDet_(—) b=(b1−wm)−(b2+wm)=(b1−b2)−2*wm  (15)

Det_a refers to watermark detection processing of the ‘b’ channel.Equation 14 and 15 are then subtracted from each other as shown below inequation 16.

$\begin{matrix}\begin{matrix}{{{Det\_ a} - {Det\_ b}} = {\left( {{a\; 1} - {a\; 2} + {2*{wm}}} \right) -}} \\{\left( {{b\; 1} - {b\; 2} - {2*{wm}}} \right)} \\{= {\left( {{a\; 1} - {a\; 2}} \right) - \left( {{b\; 1} - {b\; 2}} \right) + {4*{wm}}}}\end{matrix} & (16)\end{matrix}$

In general, related (but not necessarily immediately adjacent) frameswill have spatially correlated content. Because of the spatialcorrelation between the ‘a’ and ‘b’ frames, the image content is reducedwhile the watermark signal is reinforced. See equation 16.

For any one pair of frames selected by a watermark detector, thepolarity of the watermark could be either positive or negative. To allowfor this, the watermark detector may examine both polarities.

Improving Watermark Imperceptibility

With reference to FIG. 12, two watermark signals (or components), W₁ andW₂, are shown relative to two video frames (f₁ and f₂) over time. Ofcourse, video will likely include many more frames, and illustration ofjust a portion of such frames is not intended to be limiting.

W₁ and W₂ preferably carry the same payload or message. In terms ofsignal characteristics, however, W₁ and W₂ are preferably inverselyrelated to one another. For example, their signal polarity is inverselyrelated. Instead of two (2) watermark signals, a single watermark signalcan be used. When using a single signal, however, the signal polarity ispreferably inversely alternated between video frames.

The human eye performs temporal averaging of the watermark signals W₁and W₂ as they are rendered for viewing. That is, when looking atsequential presentment of frames f1 and f2, the human eye/mind averagesthe two signals, effectively canceling them out, since they includeinversely related polarities. Another way to view this effect is toconsider signal adjustments or “tweaks”. Recall from above that adigital watermark signal can be introduced into media content byaltering data representing audio or video or imagery. If W₁ introduces apositive (+) tweak or alteration in f1, then to achieve the favorabletemporal averaging, W₂ preferably introduces a corresponding negative(−) tweak or alteration in f2. These changes are preferably consistentfrom the first frame to the second frame. That is, if watermark changesare introduced to a first spatial area (or first set of coefficients) inthe first frame, an inverse change is made to a corresponding spatialarea (or coefficients) in the second frame.

Thus, the perceived perceptibility with temporal averaging=W₁−W₂≈0.

A programmed electronic processor (or multiple processors) embeds videoaccordingly.

A watermark detector is used to read the watermarking as watermarkedvideo is rendered. For example, the watermarked video is rendered on adisplay such as a computer monitor, TV or cell phone display (e.g.,Apple's iPhone). A camera or video camera can be used to capture imagery(e.g., streaming mode capture). Captured imagery is provided to awatermark detector which analyses captured imagery. For example, thewatermark detector preferably analyzes a single video frame to decodethe watermarking there from.

A mobile device (e.g., an iPhone) executing a watermark detectorapplication may be used for such a watermark detector. Mobile devicesare increasingly shipped with high quality video cameras. Of course,there are many other suitable devices besides the iPhone that can serveas watermark detectors.

Additional methods are now provided to improve a user's experience whentrying to read a watermark from displayed or rendered video.

A user points a camera (e.g., included in a mobile device such as theiPhone) at a display and starts video capture (see FIG. 13). There is a“sweet” spot in terms of image capture distance and image captureperspective angle to position the camera relative to the display toavoid positional distortion. If the camera is positioned too far away orat too great an angle relative to the display then the watermarking maynot be detectable. Distance and angle introduce signal distortion (e.g.,scale, rotation, translation) which may deter watermark reading.

One approach is to increase a watermark's tolerance to image capturerange and perspective angle. A watermark can be adjusted to increasedetection tolerances. For example, an embedding area or “bump” can beadjusted to allow for increased image capture range. (An embedding areaor bump refers to a host signal area or other characteristics at which asignal alteration is introduced to represent at least some portion of awatermark signal.)

If an embedding bump covering a 2×2 pixel area (4 pixels) corresponds toa sweet spot range of 4-8 inches, then doubling the bump size eightpixels to will increase the sweet spot range to about 16 inches. (Thespecific number and embedding details are not intended to be limiting,and are provided as examples only.)

Multiple bump sizes can be used when embedding watermarking in a videosequence to provide an extended image capture range. For example,consider the following frames and bump sizes:

Frame 1 Frame 2 Frame 3 Frame 4 Frame 5 Frame 6 Frame 7 Frame 8 Bump 1Bump 1 Bump 2 Bump 2 Bump 1 Bump 1 Bump 2 Bump 2

A first pair of frames is embedded at a first bump size, and a secondpair of frames is embedded at a second bump size, and so on. This willallow an extended image capture range corresponding to both bump sizesas the video is rendered. At a frame rate of 25 frames/second or higher,the user will experience quicker detection rates and lower frustrationas she positions the camera relative to the display.

Of course, three or more embedding bump sizes may be used to evenfurther extend the image capture range. And, three or more frames may begrouped together instead of using pairs.

Now let's consider image capture perspective (see FIG. 13). The idealimage capture would be a parallel vantage point directly in front of thedisplay screen. But the ideal is not always possible. For example, auser may hold their camera at an angle relative to the display whencapturing video of the display. This may introduce distortion which mayhamper watermark detection.

One approach is to precondition the watermark signal to allow for awider range of perspective angle image capture while still allowing forwatermark detection. For example, if a watermark signal can typically beread over the range of +/−5 degrees perspective angle distortion,preconditioning the watermark signal prior to embedding to +/−10 degreeson some video frames allows the image capture perspective range to beextended to approximately +/−15 degrees. For example a perspectivefilter can be set to a certain horizontal perspective, and the watermarksignal may be passed through the filter.

By way of example, please consider a watermark signal represented by aset of, e.g., horizontal lines as shown in FIG. 14. This signal is takenand preconditioned to approximate a horizontal distortion of −10 degreesas shown in FIG. 15. When this preconditioned watermark signal is viewedby a camera at a perspective angle of +10 degrees, the resultantcaptured image approximates the original signal in FIG. 14. That is, thepreconditioning coupled with an off-center read effectively cancels outthe perspective angle distortion.

By way of example, this precondition can be alternated in frames asshown below:

Frame 1 Frame 2 Frame 3 Frame 4 Frame 5 Frame 6 Frame 7 Frame 8 No No−10 −10 +10 +10 No No change change degrees degrees degrees degreeschange change

A first pair of frames is embedded without any preconditioning, a secondpair of frames is embedded with a precondition signal at −10 degreesperspective, and a third pair of frames is embedded with apreconditioned signal at +10 degrees perspective, and so on. This willallow an extended image capture perspective angle range as the video isrendered. At a frame rate of 25 frames/second or higher, the user willexperience quicker detection rates and lower frustration as shepositions the camera relative to the display.

Of course, additional angle preconditioning can be used, with differentembedding intervals, over a different number of framepairs/triplets/quads, etc. as well. Also, while we have used ±10degrees, the preconditioning can cover a range of values, e.g., over±5±20 degrees.

Moreover, both perspective preconditioning and embedding bump sizes canbe combined to yield both increased range and perspective changes. Forexample:

1^(st) Frame Pair 2^(nd) Frame Pair 3^(rd) Frame Pair 4^(th) Frame Pair5^(th) Frame Pair 6^(th) Frame Pair Bump size 1; Bump size 1; Bump size1; Bump size 2; Bump size 2; Bump size 1; No +10 degrees −10 degrees No+10 degrees −10 degrees precondition precondition preconditionprecondition precondition precondition

Of course, other combinations involving both bump size and preconditionscan be used as well. In some cases the so-called “I” frames are used asa starting reference for bump size and/or preconditioning. Sequences canbe renewed or altered when an I frame is encountered.

As discussed further in assignee's U.S. patent application Ser. No.12/640,386 (now U.S. Pat. No. 8,175,617), hereby incorporated herein byreference in its entirety, target patterns may be included in a scenefrom which, e.g., the distance to, and orientation of, surfaces withinthe viewing space can be discerned. Such targets thus serve as beacons,signaling distance and orientation information to a camera system. Onesuch target is the TRIPcode, detailed, e.g., in de Ipiña, TRIP: aLow-Cost Vision-Based Location System for Ubiquitous Computing, Personaland Ubiquitous Computing, Vol. 6, No. 3, May, 2002, pp. 206-219.

As detailed in the Ipiña paper, the target (shown in FIG. 16) encodesinformation including the target's radius, allowing a camera-equippedsystem to determine both the distance from the camera to the target, andthe target's 3D pose. If the target is positioned on a surface in theviewing space (e.g., on a wall), the Ipiña arrangement allows acamera-equipped system to understand both the distance to the wall, andthe wall's spatial orientation relative to the camera.

It may be advantageous to conceal the presence of such TRIPcodes. Onecamouflage method relies on the fact that color printing is commonlyperformed with four inks: cyan, magenta, yellow and black (CMYK).Normally, black material is printed with black ink. However, black canalso be imitated by overprinting cyan and magenta and yellow. To humans,these two techniques are essentially indistinguishable. To a digitalcamera, however, they may readily be discerned. This is because blackinks typically absorb a relatively high amount of infrared light,whereas cyan, magenta and yellow channels do not.

The arrangement just described can be adapted for use with any colorprinted imagery—not just black regions. Details for doing so areprovided in U.S. Pat. No. 7,738,673, which is hereby incorporated hereinby reference in its entirety. By such arrangements, TRIPcode targets canbe concealed or hidden from human view wherever printing may appear in avisual scene, allowing accurate measurement of certain features andobjects within the scene by reference to such targets.

A hidden TRIPcode may be advantageously used to improve watermarkdetection. For example, a hidden TRIPcode may be encoded in video priorto rendering on a display. A mobile phone or other camera captures videoof the display rendering the encoded video. The mobile phone analyzesthe captured video to discern details from the TRIPcode. These detailsinclude information to allow the mobile phone to discern an imagecapture range and image capture perspective angle. Armed with thisinformation, the mobile phone warps (e.g., alters or transforms) thecaptured video to compensate for image capture distance and perspectiveangle. This warping counteracts the effects of distance and perspectivechanges. This warped video is then provided to a watermark detector.From the watermark detector's perspective, the video is (relatively)distortion free. The detector searches for a watermark in the warpedvideo.

Although it could be, a TRIPcode (or other target pattern) need not beplaced in every video frame. In fact, such a tool could be placed every10 or more frames. In one embodiment, the watermark detection processesis not initiated until a TRIPcode (or other target pattern) is found.The video data is warped to compensate for distortion, and the warpedvideo is then presented to the watermark detector.

Concluding Remarks

Having described and illustrated the principles of the technology withreference to specific implementations, it will be recognized that thetechnology can be implemented in many other, different, forms. Toprovide a comprehensive disclosure without unduly lengthening thespecification, applicant hereby incorporates by reference each of theabove referenced patent documents in its entirety. Such documents areincorporated in their entireties, even if cited above in connection withspecific of their teachings. These documents disclose technologies andteachings that can be incorporated into the arrangements detailedherein, and into which the technologies and teachings detailed hereincan be incorporated.

The methods, processes, components, apparatus and systems describedabove may be implemented in hardware, software or a combination ofhardware and software. For example, the watermark encoding processes andembedders may be implemented in software, firmware, hardware,combinations of software, firmware and hardware, a programmablecomputer, electronic processing circuitry, and/or by executing softwareor instructions with a processor or circuitry. Similarly, watermark datadecoding or decoders may be implemented in software, firmware, hardware,combinations of software, firmware and hardware, a programmablecomputer, electronic processing circuitry, and/or by executing softwareor instructions with a multi-purpose electronic processor, parallelprocessors or cores, and/or other multi-processor configurations.

The methods and processes described above (e.g., watermark embedders anddetectors) also may be implemented in software programs (e.g., writtenin C, C++, Visual Basic, Java, Python, Tcl, Perl, Scheme, Ruby,executable binary files, etc.) stored in memory (e.g., a computerreadable medium, such as an electronic, optical or magnetic storagedevice) and executed by an electronic processor (or electronicprocessing circuitry, hardware, digital circuit, etc.).

While one embodiment discusses inverting the polarity in a second colorchannel (e.g., a ‘b’ channel), one could also invert the polarity in thefirst color channel (e.g., an ‘a’ channel) instead. In such a case, thefirst color channel is then preferably subtracted from the second colorchannel.

The particular combinations of elements and features in theabove-detailed embodiments are exemplary only; the interchanging andsubstitution of these teachings with other teachings in this and theincorporated-by-reference patents are also contemplated.

What is claimed is:
 1. An apparatus comprising: memory for storing data representing video; one or more electronic processors programmed for: embedding a first watermark signal in a first portion of the data, the first watermark signal comprising a first signal polarity and corresponding to first detection preconditioning; embedding a second watermark signal in a second portion of the data, the second watermark signal comprising a second signal polarity that is inversely related to the first signal polarity and corresponding to second detection preconditioning; controlling provision of the watermarked video for display in real time, in which temporal averaging of the first watermark signal and second watermark signal over time conceals the first watermark signal and the second watermark signal from a human observer of the video.
 2. The apparatus of claim 1 in which the first watermark signal and the second watermark signal are embedded in corresponding spatial positions of the video.
 3. A mobile device comprising the apparatus of claim
 1. 4. An apparatus comprising: memory for storing data representing video; one or more processors programmed for: embedding a watermark signal in a first portion of the data, the embedding the watermark signal in the first portion of the data using a first embedding bump size; embedding a watermark signal in a second portion of the data, the embedding the watermark signal in the second portion of the data using a second embedding bump size, in which the first embedding bump size corresponds with a first detection range distance when capturing optical scan data associated with the video as it is being rendered on a display, and the second embedding bump size corresponds with a second, larger detection range distance when capturing optical scan data associated with the video as it is being rendered on the display; controlling provision of the watermarked video to a display.
 5. The apparatus of claim 4 in which use of both the first embedding bump size and the second embedding bump size extends the detection range distance when capturing optical scan data associated with the rendered video.
 6. The apparatus of claim 4 in which the optical scan data comprises video data captured from a display that is rendering the video.
 7. The apparatus of claim 4 in which the first portion of the data comprises two or more video frames.
 8. The apparatus of claim 7 in which the second portion of the data comprises two or more video frames.
 9. A mobile device comprising the apparatus of claim
 4. 10. An apparatus comprising: memory for storing a watermark signal; one or more electronic processors programmed for: embedding a watermark signal in a first portion of a video signal; preconditioning the watermark signal in a first manner to allow expanded detection of the preconditioned watermark signal in the presence of first distortion; embedding the watermark signal preconditioned in the first manner in a second portion of the video signal; preconditioning the watermark signal in a second manner to allow expanded detection of the preconditioned watermark signal in the presence of second distortion; embedding the watermark signal preconditioned in the second manner in a third portion of the video signal.
 11. The apparatus of claim 10 in which the first distortion is due to positive image capture perspective angle variance.
 12. The apparatus of claim 11 in which the second distortion is due to negative image capture perspective angle variance.
 13. The apparatus of claim 12 in which image capture perspective angle variance is introduced by a mobile device capturing imagery of a display that is rendering the watermarked video signal.
 14. The apparatus of claim 11 in which image capture perspective angle variance is introduced by a mobile device capturing imagery of a display that is rendering the watermarked video signal.
 15. The apparatus of claim 10 in which embedding the watermark signal preconditioned in the first manner in a second portion of the video signal uses a first embedding bump size; and in which embedding the watermark signal preconditioned in the second manner in a third portion of the video signal uses a second embedding bump size; and in which the first embedding bump size corresponds with a first optimal detection range distance when capturing optical scan data associated with the video signal as it is being rendered on a display, and the second embedding bump size corresponds with a second, larger optimal detection range distance when capturing optical scan data associated with the video signal as it is being rendered on the display.
 16. A mobile device comprising the apparatus of claim
 10. 17. A non-transitory computer readable medium comprising instructions stored thereon to cause one or more processors to perform as follows: embedding a watermark signal in a first portion of the data, the embedding the watermark signal in the first portion of the data using a first embedding bump size; embedding a watermark signal in a second portion of the data, the embedding the watermark signal in the second portion of the data using a second embedding bump size, in which the first embedding bump size corresponds with a first detection range distance when capturing optical scan data associated with the video as it is being rendered on a display, and the second embedding bump size corresponds with a second, larger detection range distance when capturing optical scan data associated with the video as it is being rendered on the display; controlling provision of the watermarked video to a display.
 18. The non-transitory computer readable medium of claim 17 in which use of both the first embedding bump size and the second embedding bump size extends the detection range distance when capturing optical scan data associated with the rendered video.
 19. The non-transitory computer readable medium of claim 17 in which the optical scan data comprises video data captured from a display that is rendering the video.
 20. The non-transitory computer readable medium of claim 17 in which the first portion of the data comprises two or more video frames.
 21. The non-transitory computer readable medium of claim 20 in which the second portion of the data comprises two or more video frames.
 22. A method comprising: obtaining data representing video; embedding a watermark signal in a first portion of the data, the embedding the watermark signal in the first portion of the data using a first embedding bump size; embedding a watermark signal in a second portion of the data, the embedding the watermark signal in the second portion of the data using a second embedding bump size, in which the first embedding bump size corresponds with a first detection range distance when capturing optical scan data associated with the video as it is being rendered on a display, and the second embedding bump size corresponds with a second, larger detection range distance when capturing optical scan data associated with the video as it is being rendered on the display; controlling provision of the watermarked video to a display.
 23. The method of claim 22 in which use of both the first embedding bump size and the second embedding bump size extends the detection range distance when capturing optical scan data associated with the rendered video.
 24. The method of claim 22 in which the optical scan data comprises video data captured from a display that is rendering the video.
 25. The method of claim 22 in which the first portion of the data comprises two or more video frames.
 26. The method of claim 25 in which the second portion of the data comprises two or more video frames.
 27. A non-transitory computer readable medium comprising instructions stored thereon to cause one or more processors to perform as follows: embedding a watermark signal in a first portion of a video signal; preconditioning the watermark signal in a first manner to allow expanded detection of said preconditioned watermark signal in the presence of first distortion; embedding the watermark signal preconditioned in the first manner in a second portion of the video signal; preconditioning the watermark signal in a second manner to allow expanded detection of said preconditioned watermark signal in the presence of second distortion; embedding the watermark signal preconditioned in the second manner in a third portion of the video signal.
 28. The non-transitory computer readable medium of claim 27 in which the first distortion is due to positive image capture perspective angle variance.
 29. The non-transitory computer readable medium of claim 28 in which the second distortion is due to negative image capture perspective angle variance.
 30. The non-transitory computer readable medium of claim 29 in which image capture perspective angle variance is introduced by a mobile device capturing imagery of a display that is rendering the watermarked video signal.
 31. The non-transitory computer readable medium of claim 28 in which image capture perspective angle variance is introduced by a mobile device capturing imagery of a display that is rendering the watermarked video signal.
 32. The non-transitory computer readable medium of claim 27 in which embedding the watermark signal preconditioned in the first manner in a second portion of the video signal uses a first embedding bump size; and in which embedding the watermark signal preconditioned in the second manner in a third portion of the video signal uses a second embedding bump size; and in which the first embedding bump size corresponds with a first optimal detection range distance when capturing optical scan data associated with the video signal as it is being rendered on a display, and the second embedding bump size corresponds with a second, larger optimal detection range distance when capturing optical scan data associated with the video signal as it is being rendered on the display.
 33. A method comprising: embedding a watermark signal in a first portion of a video signal; preconditioning the watermark signal in a first manner to allow expanded detection of said preconditioned watermark signal in the presence of first distortion; embedding the watermark signal preconditioned in the first manner in a second portion of the video signal; preconditioning the watermark signal in a second manner to allow expanded detection of said preconditioned watermark signal in the presence of second distortion; embedding the watermark signal preconditioned in the second manner in a third portion of the video signal.
 34. The method of claim 33 in which the first distortion is due to positive image capture perspective angle variance.
 35. The method of claim 34 in which the second distortion is due to negative image capture perspective angle variance.
 36. The method of claim 35 in which image capture perspective angle variance is introduced by a mobile device capturing imagery of a display that is rendering the watermarked video signal.
 37. The method of claim 34 in which image capture perspective angle variance is introduced by a mobile device capturing imagery of a display that is rendering the watermarked video signal.
 38. The method of claim 33 in which embedding the watermark signal preconditioned in the first manner in a second portion of the video signal uses a first embedding bump size; and in which embedding the watermark signal preconditioned in the second manner in a third portion of the video signal uses a second embedding bump size; and in which the first embedding bump size corresponds with a first optimal detection range distance when capturing optical scan data associated with the video signal as it is being rendered on a display, and the second embedding bump size corresponds with a second, larger optimal detection range distance when capturing optical scan data associated with the video signal as it is being rendered on the display. 